Stopping Times

Stopping times

Let (\Omega, \sF, P) be a probability space. Consider a random process X:\Omega\to\sX^T defined on this probability space with state space \sX \subseteq \R and ordered index set T\subseteq \R considered as time.

Definition 1. A collection of event spaces denoted \sG_{\bullet} \triangleq (\sG_t \subseteq \sF: t \in T) is called a filtration if \sG_s \subseteq \sG_t for all s \le t.

Remark 1. For the random process X:\Omega\to\sX^T, we can find the event space generated by all random variables until time t as \sF_t\triangleq \sigma(X_s, s \le t). The collection of event spaces \sF_\bullet \triangleq (\sF_t: t \in T) is a filtration.

Definition 2. The natural filtration associated with a random process X:\Omega \to \sX^T is given by \sF_\bullet \triangleq (\sF_t: t\in T) where \sF_t \triangleq \sigma(X_s, s\le t).

Remark 2. For a random sequence X: \Omega \to \sX^\N, the natural filtration is a sequence \sF_\bullet = (\sF_n: n \in \N) of event spaces \sF_n \triangleq \sigma(X_1, \dots, X_n) for all n \in \N.

Remark 3. If the random sequence X is independent, then the random sequence (X_{n+j}: j \in \N) is independent of the event space \sigma(X_1, \dots, X_n).

Example 3. For a random walk S with step size sequence X, the natural filtration of the random walk is identical to that of the step size sequence. That is, \sigma(S_1, \dots, S_n) = \sigma(X_1, \dots, X_n) for all n \in \N. This follows from the fact that for all n \in \N, we can can write S_j = \sum_{i=1}^jX_i and X_j = S_j - S_{j-1} for all j \in [n]. That is, there is a bijection between (X_1, \dots, X_n) and (S_1, \dots, S_n).

Definition 4. A random variable \tau: \Omega \to T is called a stopping time with respect to a filtration \sF_\bullet if

1. the event \tau^{-1}(-\infty, t] \in \sF_t for all t \in T, and

2. the random variable \tau is finite almost surely, i.e. P\set{\tau < \infty} = 1.

Remark 4. Intuitively, if we observe the process X sequentially, then the event \set{\tau \le t} can be completely determined by the observation (X_s, s \le t) until time t. The intuition behind a stopping time is that its realization is determined by the past and present events but not by future events. That is, given the history of the process until time t, we can tell whether the stopping time is less than or equal to t or not. In particular, \E[\SetIn{\tau \le t}\given \sF_t]=\SetIn{\tau \le t} is either one or zero.

Definition 5 (First hitting time). For a process X:\Omega\to\sX^T and any Borel measurable set A \in \sB(\sX), we define the first hitting time \tau_X^A: \Omega \to T\cup\set{\infty} for the process X to hit states in A, as \tau_X^A \triangleq \inf\set{t \in T: X_t \in A}.

Example 6. We observe that the event \set{\tau_X^A \le t} = \set{X_s \in A \text{ for some }s \le t} \in\sF_t for all t \in T. It follows that, if \tau_A is finite almost surely, then \tau_A is a stopping time with respect to filtration \sF_\bullet.

Proposition 7. For a random sequence X: \Omega \to \sX^\N, an almost sure finite discrete random variable \tau: \Omega \to \N\cup\set{\infty} is a stopping time with respect to this random sequence X iff the event \set{\tau = n} \in \sigma(X_1, \dots, X_n) for all n \in \N.

Proof. Proof. From Definition [defn:StoppingTimeGen], we have \set{\tau = n} = \set{\tau \le n}\setminus\set{\tau \le n-1} \in \sigma(X_1, \dots, X_n). Conversely, from the theorem hypothesis, it follows that \set{\tau \le n} = \cup_{m=1}^n\set{\tau = m} \in \sigma(X_1, \dots, X_n). ◻

Example 8. Consider a random sequence X: \Omega \to \sX^\N, with the natural filtration \sF_\bullet, and a measurable set A \in \sB(\sX). If the first hitting time \tau_X^A:\Omega\to\N\cup\set{\infty} for the sequence X to hit set A is almost surely finite, then \tau_X^A is a stopping time. This follows from the fact that \set{\tau_X^A = n} = \cap_{k=1}^{n-1}\set{X_k \notin A}\cap\set{X_n \in A} \in \sF_n for each n \in \N.

Definition 9. Consider a random process X:\Omega\to \sX^{\R_+} with discrete state space \sX\subseteq\R. For each state y \in \sX, we define \tau_X^{\set{y},0} \triangleq 0 and inductively define the kth hitting time to a state y after time t = 0, as

Remark 5. We observe that \set{\tau_X^{\set{y}, k} \le t} \in \sF_t for all times t \in \R_+. Hence if \tau_X^{\set{y}, k} is almost surely finite, then it is a stopping time for the process X.

Definition 10. For a discrete valued random sequence X:\Omega\to\sX^\N, the number of visits to a state y \in \sX in first n time steps is defined as N_y(n) \triangleq \sum_{k=1}^n\SetIn{X_k = y} for all n \in \N.

Remark 6. We observe that N_y:\Omega\to\Z_+^{\Z_+} is a random walk with the Bernoulli step size sequence (\SetIn{X_k = y}: k \in \N). Further, \tau_X^{\set{y}, k} = \tau_{N_y}^{\set{k}} = \inf\set{n \in \N: N_y(n) = k}. We also observe that \{N_y(n) \le k\} = \{\tau_y^{(k+1)} > n\} and \{N_y(n) = k\} = \{\tau_y^{k} \le n < \tau_y^{(k+1)}\}.

Remark 7. We observe that the number of visits to state y in first n steps of X is also given by This implies that N_y(n)+1 is the first hitting time to set of states \set{n+1, n+2, \dots } for the increasing random sequence (\tau_X^{\set{y},k}:k \in \N).

Lemma 11 (Wald’s Lemma). Consider a random walk S: \Omega \to \R^{\Z_+} with step-sizes X: \Omega \to \R^\N having finite \E\abs{X_1}. Let \tau be a finite mean stopping time with respect to this random walk. Then,

Proof. Proof. Recall that the event space generate by the random walk and the step-sizes are identical. From the independence of step sizes, it follows that X_n is independent of \sigma(X_0, X_1, \dots, X_{n-1}). Since \tau is a stopping time with respect to random walk S, we observe that \set{\tau \ge n} = \set{\tau > n-1} \in \sigma(X_0, X_1, \dots, X_{n-1}), and hence it follows that random variable X_n and indicator \SetIn{\tau \ge n} are independent and \E[X_n\SetIn{\tau \ge n}] = \E X_1 \E\SetIn{\tau \ge n}. Therefore, We exchanged limit and expectation in the above step, which is not always allowed. We were able to do it by the application of dominated convergence theorem. ◻

Corollary 12. Consider the stopping time \tau_S^{\set{i}} \triangleq \min\set{n \in \N: S_n = i} for an integer random walk S: \Omega \to \Z_+^{\Z_+} with step size sequence X: \Omega \to \Z^\N. Then, the mean of stopping time \E \tau_S^{\set{i}} = {i}/{\E X_1}.

Proof. Proof. This follows from the Wald’s Lemma and the fact that S_{\tau_i} = i. ◻

Properties of stopping time

Lemma 13. Let \tau_1,\tau_2 be two stopping times with respect to filtration \sF_\bullet. Then the following hold true.

1. \min \set{\tau_1,\tau_2} is a stopping time.

2. If T is separable, then \tau_1+\tau_2 is a stopping time.

Proof. Proof. Let \tau_1,\tau_2 be stopping times with respect to a filtration \sF_{\bullet} = (\sF_t : t \in T).

1. Result follows since the event \set{ \min \set{ \tau_1,\tau_2} > t} = \set{\tau_1 > t} \cap \set{\tau_2 > t} \in \sF_t.

2. A topological space is called separable if it contains a countable dense set. Since \R_+ is separable and ordered, we assume T = \R_+ without any loss of generality. It suffices to show that the event \set{\tau_1+\tau_2 \le t} \in \sF_t for T = \R_+. To this end, we observe that

 ◻

Strong independence property and applications

Theorem 14 (Strong independence property). Let X: \Omega\to\sX^{\R_+} be an independent random process with natural filtration \sF_\bullet, and \tau:\Omega\to\R_+ a stopping time with respect to \sF_\bullet, then (X_{\tau+s}: s \in \R_+) is independent of history (X_s: s \le \tau).

Definition 15. We can define the kth return time to state y for the random process X:\Omega\to\sX^{\R_+} as the interval between two successive visits to state y, that is for all k \in \N

Remark 8. We observe that \tau_X^{\set{y}, k} = \sum_{j=1}^kH_X^{\set{y}, j} for all k \in \N. That is, the hitting times sequence (\tau_X^{\set{y},k}: k \in \N) is a random walk with step-size sequence being return times (H_X^{\set{y},j}: j \in \N). Therefore, if H_X^{\set{y}, j} is almost sure finite for all j \in [k], then the finite sum \tau_X^{\set{y}, k} is almost sure finite for all k\in \N.

Remark 9. From the bijection between hitting and return times, we observe that \sigma(\tau_X^{\set{y}, k)}: k \in [n]) = \sigma(H_X^{\set{y}, j}: j \in [n]). Recall that \tau_X^{N_y(n)+1} > n by definition. If passage times are independent and from second passage time, then it follows from the Wald’s Lemma that

Example 16. We also observe that the kth hitting time to \set{1} by a Bernoulli step size sequence X:\Omega\to\set{0,1}^\N is the first hitting time to \set{k} by random walk S:\Omega\to\Z_+^{\Z_+}. That is,

Lemma 17. For an Bernoulli random sequence X:\Omega\to\set{0,1}^\N with \E X_1 \in (0,1), the kth hitting time to state 1 is a stopping time, and \tau_X^{\set{1}, k} = \sum_{i=1}^kY_i where Y:\Omega\to\N^\N is an random sequence distributed identically to \tau_X^{\set{1}} .

Proof. Proof. When the Bernoulli step size sequence X is with \E X_1 = p \in (0,1), we get that P\set{\tau_S^{\set{1}}= n} = (1-p)^{n-1}p for all n \in \N. It follows that Hence, the random time \tau_S^{\set{1}} is finite almost surely. We will show that \tau_S^{\set{k}} is finite almost surely for all k \in \N by induction. By induction hypothesis, \tau_S^{\set{k-1}} is finite almost surely. Then S_{\tau_S^{\set{k-1}}+n}-S_{\tau_S^{\set{k-1}}} = \sum_{j=1}^nX_{\tau_S^{\set{k-1}}+j} is the sum of n Bernoulli random variables independent of \tau_S^{\set{k-1}} by strong independence property, and hence has distribution identical to S_n. Further, This implies that \tau_S^{\set{k}} = \tau_S^{\set{k-1}} + \tau_S^{\set{1}}, where \tau_S^{\set{1}} has the identical distribution to \tau_X^{\set{1}} and is independent of \tau_S^{\set{k-1}}. ◻